Multimodal Facial Feature Extraction for Automatic 3D Face Recognition

Facial feature extraction is important in many face-related applications, such as face alignment for recognition. We propose a multimodal scheme to integrate 3D (range) and 2D (intensity) information provided from a facial scan to extract the feature points. Given a face scan, the foreground is segmented from the background using the range map and the face area is detected using a real-time intensity-based algorithm. A robust nose tip locator is presented. A statistical 3D feature location model is applied after aligning the model with the nose tip. The shape index response derived from the range map and the cornerness response from the intensity map are combined to determine the positions of the corners of the eyes and the mouth. Real-world data is subject to sensor noise, resulting in spurious feature points. We introduce a local quality metric to automatically reject the scan whose sensor noise is above a certain threshold. As a result, a fully automatic multimodal face recognition system is developed. Both qualitative and quantitative evaluations are conducted for the proposed feature extraction algorithm on a publicly available database, containing 946 facial scans of 267 subjects. This automatic feature extraction algorithm has been integrated in an automatic face recognition system. The identification performance on a database of 198 probe scans and 200 gallery subjects is close to that with manually labeled landmarks.